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Abstract. We consider a zero-sum stochastic differential controUer-and-stopper game in which 
the state process is a controlled diffusion evolving in a multi-dimensional Euclidean space. In this 
game, the controller affects both the drift and diffusion terms of the state process, and the diffusion 
' term can be degenerate. Under appropriate conditions, we show that the game has a value and the 

OA ■ value function is the unique viscosity solution to an obstacle problem for a Hamilton- Jacobi-Bellman 

equation. 

' Key Words: Controller-stopper games, weak dynamic programming principle, viscosity solutions, 

robust optimal stopping. 



1. Introduction 



O 
^' 

J2 ■ We consider a zero-sum stochastic differential game of control and stopping under a fixed time 
horizon T > 0. There are two players, the "controller" and the "stopper," and a state process 

_ X° which can be manipulated by the controller through the selection of the control a. Suppose 

^> I the game starts at time t £ [0,T]. While the stopper has the right to choose the duration of this 

\ game (in the form of a random time r), she incurs the running cost /(s, X", ag) at every moment 

^ I t < s < T, and the terminal cost g{X") at the time the game stops. Given the instantaneous 

^ i discount rate c{s,Xf), the stopper would like to minimize her expected discounted cost 

o: 

O- E ^ ^ 



(1.1) 



over all choices of r. At the same time, however, the controller plays against her by maximizing 
^ " (jl-ip over all choices of a. 

I Ever since the game of control and stopping was introduced by Maitra & Sudderth [25] , it has 



been known to be closely related to some common problems in mathematical finance, such as pricing 
American contingent claims (see e.g. [17 1 12 H [22]) and minimizing the probability of lifetime ruin 
(see [3]). The game itself, however, has not been studied to a great extent except certain particular 
cases. Karatzas and Sudderth [20] study a zero-sum controller-and-stopper game in which the 
state process X" is a one-dimensional diffusion along a given interval on M. Under appropriate 
conditions they prove that this game has a value and describe fairly explicitly a saddle point of 
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optimal choices. It turns out, however, difficult to extend their results to multi-dimensional cases, 
as their techniques rely heavily on theorems of optimal stopping for one-dimensional diffusions. To 
deal with zero-sum multi-dimensional games of control and stopping, Karatzas and Zamfirescu [23] 
develop a martingale approach; also see [2], [4] and [3]. Again, it is shown that the game has a 
value, and a saddle point of optimal choices is constructed. However, it is assumed to be that the 
controller can affect only the drift term of X". 

There is yet another subtle discrepancy between the one-dimensional game in [20] and the multi- 
dimensional game in |23] : the use of "strategies". Typically, in a two-player game, the player who 
acts first would not choose a fixed static action. Instead, she prefers to employ a strategy, which 
will give different responses to different future actions the other player will take. This additional 
flexibility enables the player to further decrease (increase) the expected cost, if she is the minimizer 
(maximizer). For example, in a game with two controllers (see e.g. |13[|12[[T^ [m[7]). the controller 
who acts flrst employs a strategy, which is a function that takes the other controller's latter decision 
as input and generates a control. Note that the use of strategies is preserved in the one-dimensional 
controller-and-stopper game in [20]: what the stopper employs is not simply a stopping time, but 
a strategy in the form of a random time which depends on the controller's decision. This kind 
of dynamic interaction is missing, however, in the multi-dimensional case: in [23], the stopper is 
restricted to use stopping times, which give the same response to any choice the controller makes. 

Zero-sum multi-dimensional controller-and-stopper games are also covered in Hamadene &: Lep- 
eltier |16| and Hamadene |15] . as a special case of mixed games introduced there. The main tool 
used in these papers is the theory of backward differential equations with two reflecting barriers. 
Interestingly, even though the method in I15j differs largely from that in [23], these two papers 
also require a diffusion coefficient which is not affected by the controller, and do not allow the use 
of strategies. This is in contrast with the one-dimensional case in |20) . where everything works out 
fine without any of the above restrictions. It is therefore of interest to see whether we can construct 
a new methodology under which multi-dimensional controller-and-stopper games can be analyzed 
even when the conditions required in [231 HI IIS] foil to hold. 

In this paper, such a methodology is built, under a Markovian framework. On the one hand, 
we allow both the drift and diffusion terms of the state process X" to be controlled. On the 
other hand, we allow the players to use strategies. Specifically, we first define non- anticipating 
strategies in Definition 13.11 Then, in contrast to two-controller games where both players use 
strategies, only the stopper chooses to use strategies in our case (which coincides with the set-up 
in [20]). This is because by the nature of a controller-and-stopper game, the controller cannot 
benefit from using non-anticipating strategies; see Remark 13.21 With this observation in mind, 
we give appropriate definitions of the upper value function U and the lower value function V in 
(|3.5|) and (|3.6p respectively. Under this set-up, we show that e-optimal saddle points always exist 
(Proposition 13. 2[ ). However, we are not interested in imposing additional assumptions in order to 
construct a saddle point (as is done in [2D1 [231 [T21 [TB] ; see Remark |3. 5 1) . Instead, we intend to work 
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under a rather general framework, and determine under what conditions the game has a value (i.e. 
U = V) and how we can derive a PDE characterization for this value when it exists. 

Our method is motivated by Bouchard &: Touzi [8], where the weak dynamic programming 
principle for stochastic control problems was first introduced. By generalizing the weak dynamic 
programming principle in [8] to the context of controller- and-stopper games, we show that V is a 
viscosity supersolution and U* is a viscosity subsolution to an obstacle problem for a Hamilton- 
Jacobi-Bellman equation, where U* denotes the upper semicontinuous envelope of U defined as in 
(|1.2|) . More specifically, we first prove a continuity result for an optimal stopping problem embed- 
ded in V (Lemma 14. ip . which enables us to follow the arguments in [HI Theorem 3.5] even under 
the current context of controller-and-stopper games. We obtain, accordingly, a weak dynamic pro- 
gramming principle for V (Proposition 14. ip , which is the key to proving the supersolution property 
of V (Propositions . On the other hand, by generalizing the arguments in Chapter 3 of Krylov 
|24] , we derive a continuity result for an optimal control problem embedded in U (Lemma 15. 4p . 
This leads to a weak dynamic programming principle for U (Proposition 15. ip , from which the sub- 
solution property of U* follows (Proposition 15. 2p . Finally, under appropriate conditions, we prove 
a comparison result for the associated obstacle problem. Since ^ is a viscosity supersolution and 
U* is a viscosity subsolution, the comparison result implies U* < V. Recalling that U* is actually 
larger than V by definition, we conclude that U* = V. This in particular implies U = V, i.e. the 
game has a value, and the value function is the unique viscosity solution to the associated obstacle 
problem. This is the main result of this paper; see Theorem 16.11 Note that once we have this 
PDE characterization, we can compute the value of the game using a stochastic numerical scheme 
proposed in Bayraktar & Fahim [T]. 

Another important advantage of our method is that it does not require any non-degeneracy 
condition on the diffusion term of X". For the multi-dimensional case in \23\ [161 [15] . Girsanov's 
theorem plays a crucial role, which entails non-degeneracy of the diffusion term. Even for the one- 
dimensional case in [20] , this non-degeneracy is needed to ensure the existence of the state process 
(in the weak sense). Note that Weerasinghe [32] actually follows the one-dimensional model in 
|20j and extends it to the case with degenerate diffusion term; but at the same time, she assumes 
boundedness of the diffusion term, and some specific conditions including twice differentiability of 
the drift term and concavity of the cost function. 

It is worth noting that while |23[ \W[ [T5] do not allow the use of strategies and require the diffusion 
coefficient be control-independent and non-degenerate, they allow for non-Markovian dynamics and 
cost structures, as well as for non-Lipschitz drift coefficients. As a first step to allowing the use of 
strategies and incorporating controlled, and possibly degenerate, diffusion coefficients in a zero-sum 
multi-dimensional controller-and-stopper game, this paper focuses on proving the existence and 
characterization of the value of the game under a Markovian framework with Lipschitz coefficients. 
We leave the general non-Markovian and non-Lipschitz case for future research. 

The structure of this paper is as follows: in Section 2, we set up the framework of our study. 
In Section [3l we define strategies, give appropriate definitions of the upper value function U and 
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the lower value function V, and show the existence of e-optimal saddle points. In Sections S] and 
m the supersolution property of V and the subsolution property U* are derived, respectively. In 
Section [6l we prove a comparison theorem, which leads to the existence of the value of the game 
and the viscosity solution property of the value function. 

1.1. Notation. We collect some notation and definitions here for readers' convenience. 

• Given a probability space {E,I,P), we denote by L^{E,Z) the set of real-valued random 
variables on {E,I); for p £ [l,oo), let Ln{E,I, P) denote the set of M"-valued random 
variables R on {E,I) s.t. Ep[|i?|*'] < oo. For the "n = 1" case, we simply write as L^. 

• M+ := [0, oo) and 5 := M'' x M+ x M+. 

• M*^ denotes the set of d x d real matrices. 

• Given E C M"-, LSC(£') denotes the set of lower semicontinuous functions defined on E, 
and USC(i?) denotes the set of upper semicontinuous functions defined on E. 

• Let be a normed space. Given w : [0, T] x i— )• M, we define the upper and lower 
semicontinuous envelopes of w, respectively, by 

w*{t,x) := limsup{u;(t',x') | {t',x') G [0,r) x E with \t' - t\ < 6, \x' - x\ < 6}; 

(1.2) 

w^{t,x) := limmi{w{t',x') \ {t',x') G [0,r) x E with \t' - t\ < 6, \x' - x\ < 6}. 

2. Preliminaries 

2.1. The Set-up. Fix T > and d G N. For any t G [0,r], let il* := C{[t,T];R'^) be the canonical 
space of continuous paths equipped with the uniform norm ||w||(,t := sup^gj^ j.] \u}s\, to G fi*. Let 
VF* denote the canonical process on 0*, and G* = {Qi}se[t,T] denote the natural filtration generated 
by WK Let P* be the Wiener measure on (17*,^^), and consider the collection of P*-null sets 
TV"* := {N £ gi^\ W\N) = 0} and its completion AT* := {A C 17* | yl C Af for some G M^}. 
Now, define G* = {^l}se[t,T] the augmentation of G* by the sets in A/"*, i.e. q\ := (j{Ql U AA*), 
s G [t, T]. For any x G M'^, we also consider gl''' := Gl n {W^ = x], Vs G [t, T\. For 0*, W\ Af\ Jf\ 
Q\, q\ and Q^'^ , we drop the superscript t whenever t = Q. 

Given x G W^, we define for any G fi* the shifted path {Cj + x). := Cj. + x, and for any j4 C Jl* 
the shifted set ^ + x:={a'Gil* IlD — xG A}. Then, we define the shifted Wiener measure P*'^ by 
P*.^(F) := P*(F - x), F G and let P*'"" denote the extension of P*'^ on {n\Q%). For P*'^ and 

— t X 

P ' , we drop the superscripts t and x whenever t = and x = 0. We let E denote the expectation 
taken under P. 

Fix t G [0, r] and w G O. For any uj G J^*, we define the concatenation of oj and a; at i as 

Note that oj (^t^ lies in il. Consider the shift operator in space : fi* i— )• il* defined by ^t{^) '■= 
oj — Ot, and the shift operator in time (/)t : 17 i— )• il* defined by (pti^^) '■= ^\[t,T]i the restriction oiuj £ Q 
on [t,T]. For any r G [t,T], since ipt and (pt are by definition continuous under the norms || • \\t^r 
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and II • ||o,r respectively, ipt : (17*,^*) i— (Q*,^*) and (pt : {Q,Gr) ^ (f^*,^*) are Borel measurable. 
Then, for any ^ : $7 i— )• M, we define the shifted functions ^^'^ : J] i— )• M by 

:= ^{u M^')) for uj' e n. 
Given a random time r : O i— >• [0, oo], whenever w G i7 is fixed, we simplify our notation as 

Definition 2.1. On the space we define, for each t G [0,T], the filtration F* = {-7^J}sg[o,T] by 



{9,n}, ifs€[0,t], 
af^.-Vr'^s'^uAT) , ifse[t,T]. 



Ps '■= where jj : = 

We drop the superscript t whenever i = 0. 

Remark 2.1. Given t £ [0,T], note that is a collection of subsets of Q for each s £ [0,T], 
whereas Ql, ^* and QI''^ are collections of subsets of for each s £ [t,T]. 

Remark 2.2. By definition, Jg = Q s'i s £ [0,T]; then the right continuity of G implies J-g = Qs 
Vs G [0, T] i.e. F = G. Moreover, from, Lemma \A.l\ (Hi) in Appendix [A] and the right continuity of 
G, we see that J"* C = J"^ Vs G [0, T], i.e. F* C F. 

Remark 2.3. Intuitively, F* represents the information structure one would have if one starts 
observing at time t G [0,r]. More precisely, for any s G \t,T], Q^s^ represents the information 
structure one obtains after making observations on in the period [t,s]. One could then deduce 
from Q^s^ the information structure (p^^ip^^Qs^ for W on the interval [0,s]. 

We define 7~* as the set of all F*-stopping times which take values in [0, T] P-a.s., and At as the 
set of all F*-progressively measurable M-valued processes, where M is a separable metric space. 
Also, for any F-stopping times ti,T2 with ri < T2 P-a.s., we denote by T^^.t-j the set of all t £ 
which take values in [Ti,r2] P-a.s. Again, we drop the sub- or superscript t whenever t = 0. 

2.2. The State Process. Given {t, x) G [0, T] x R'^ and a £ A, let denote a M'^-valued 

process satisfying the following SDE: 

dX'f''' = b{s,X'f''',as)ds + a(s,X'f'",as)dWs, s £ [t,T], (2.1) 

with the initial condition x^'^'" = x. Let M'^ be the set of ci x d real matrices. We assume that 
b : [0,T] X R'^ X M ^ R"^ and a : [0, T] x M'^ x M M'^ are deterministic Borel functions, and 
b{t,x,u) and a{t,x,u) are continuous in {x,u); moreover, there exists K > such that for any 
t £ [0,T], x,y£ R'^, and u£ M, 

\b{t,x,u) - b{t,y,u)\ + \a{t,x,u) - a{t,y,u)\ < K\x - y\, (2.2) 
\b{t,x,u)\ + \a{t,x,u)\ < K{1 + \x\). (2.3) 
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The conditions above imply that: for any initial condition [t, x) G [0, T] x and control a ^ A, 
(12. 1[) admits a unique strong solution X^'^'". Moreover, without loss of generality, we define 

X*'^''" := X for s<t. (2.4) 

Remark 2.4. Fix a & A. Under (|2.2|) and (|2.3|) . i/ie same calculations in |28t Appendix] and 
Proposition 1.2.1] j/ieW i/ie following estimates: for each p > 1, there exists Cp{a) > such that 
for any {t,x), {f ,x') G [0,r] x R'^, and hG[0,T-t], 



E 



sup 



E 



sup \X, 

0<s<t+h 



0<s<T 

t,x,a 



<Cpii + \xn 

< Cph2{i + \x\py, 



E 



sup |X 

0<s<T 



t'x'a 



Y-t,x,aip 



< c„ 



\x' - x\P + \t' - t\2 {1 + \x\P) 



(2.5) 
(2.6) 
(2.7) 



Remark 2.5 (flow property). By pathwise uniqueness of the solution to (j2.ip . for any < t < 

s < T , X & W^, and a ^ A, we have the following two properties: 

(i) Xr'^^'^iuj) = X''^*'"'"'"(a;) V r G [s,T], forF-a.e. lo E see [6, Chapter 2] and [29l p.41]. 

(ii) By (1-16) in [14J and the discussion below it, forV-a.e. w G $7, we have 

X^'^'" (w ®s ^^(w')) = (w') Vr e [s,r], /or P-a.e € 

see also [271 Lemma 3.3]. 

2.3. Properties of Shifted Objects. Let us first derive some properties of J-"|,-measurable ran- 
dom variables. 

Proposition 2.1. Fixt G [0,r] and ^ G L^{n,T\,). 

(i) and Tt are independent. This in particular implies that ^ is independent of J-t- 

(ii) There exist N,M gM such that: for any fixed to £n\N, ^*''^(a;') = ^{uj') Vw' G \ M. 



Proof. See Appendix lA.ll 

Fix 6 ^T. Given q G we can define, for P-a.e. w G 0, a control a^'*^ G ^6»(aj) by 

a^'"(a;') := {af'"(u;')}rG[o,T] = {"r ®e 4>e{^'))] r^y^^r] . ^' ^ 
see [U proof of Proposition 5.4]. Here, we state a similar result for stopping times in T. 

Proposition 2.2. Fix 9 ^ T. For any r G Tg^T, we have t^'^ G T^^^-^x P-a-e. a; G il. 
Proof. See Appendix IA.2I 



□ 



□ 



Let p : M X M I— )■ M be any given metric on M. By [24^ p. 142], p'{u,v) := ;| arctan p(n, f ) < 1 
for u,v € M is a. metric equivalent to p, from which we can construct a metric on A by 

rT 



p{a,/3) :=E 



p'{at,/3t)dt 



for a, 13 G A. 



Now, we state a generalized version of Proposition 12. II (ii) for controls a G A. 
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Proposition 2.3. Fix t G [0, T] and a ^ At- There exists N £J7 such that: for any oj £ il \ N , 

p{a*'^^a) = 0. Furthermore, for any {s,x) £ [0,T] x R'^, X"'"^'"*'" (a;') = r G [s,T], for 

F-a.e. Lo' G Q. 

Proof. See Appendix IA.31 □ 

3. Problem Formulation 

We consider a controller-and-stopper game under the finite time horizon T > 0. While the 
controller has the ability to affect the state process through the selection of the control a, 
the stopper has the right to choose the duration of this game, in the form of a random time r. 
Suppose the game starts at time t G [0,T]. The stopper incurs the running cost f{s,Xg,as) at 
every moment t < s < t, and the terminal cost g{X") at the time the game stops, where / and 
g are some given deterministic functions. According to the instantaneous discount rate c{s,Xg) 
for some given deterministic function c, the two players interact as follows: the stopper would like 
to stop optimally so that her expected discounted cost could be minimized, whereas the controller 
intends to act adversely against her by manipulating the state process X" in a way that frustrates 
the effort of the stopper. 

For any t G [0, T], there are two possible scenarios for this game. In the first scenario, the stopper 
acts first. At time t, while the stopper is allowed to use the information of the path of W up to 
time t for her decision making, the controller has advantage: she has access to not only the path 
of up to t but also the stopper's decision. Choosing one single stopping time, as a result, might 
not be optimal for the stopper. Instead, she would like to employ a stopping strategy which will 
give different responses to different future actions the controller will take. 

Definition 3.1. Given t G [0, T], an admissible strategy vr on the horizon [t,T] is a function 
n : Ai-^ T^rp s.t. for any a, (3 £ A, the following holds for F-a.e. io G Q: 

if min{7r[a](cj),7r[/3](a;)} < inf{s > t \ as{uj) / /3^(a;)}, then n[a]{uj) = 7r[/3](a;). (3.1) 

We denote by j. the collection of all admissible strategies. 

Our definition of stopping strategies is reasonable in the sense that it is equivalent to the non- 
anticipativity requirement for strategies in game theory, as the following result demonstrates. 

Proposition 3.1. Fix t G [0,T]. For any function tt : A t-^ T^T' ^ ^/ '^'^'^ '^''^^U ^/ ^ ^■^ 
non- anticipating in the following sense: 

For any a, /3 £ A and s £ [t,T], l{n[a]<s} = M^[IB]<s} for F-a.e. u £ {a =[t,s) (3.2) 

where {a =[t,s) /3} := {oJ £ Cl \ ar{uj) = Pri^) for s £ [t, s)}. 

Remark 3.1. The non-anticipativity condition (j3.2p is used in [7]; see Assumption (C5) therein. 

Proof. For any a^P £ A, we set 0{uj) := inf{s > t \ as{uj) 7^ I3s{uj)}. 

Step 1: Suppose tt G Yi\rp. Take some N £N such that (iO) holds for u: £Vl\N. Fix a, /3 G ^ 
and s £ [t,T]. Given uj £ {a =[t,s) /?} \ N , we have s < Oiuj). If 7r[a](a;) < ^(w), then ()3.ip implies 
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7r[Q!](a;) = 7r[/3](6t;), and thus l{7r[a]<s}(w) = l{^[^]<s}(6t;). If 7r[a](Lt;) > 0{u}), then (j3.ip imphes 
7r[/3](a;) > 6{uj) too, and thus l{7r[a]<s}('^) = = l{7r[/3]<s}('^) as s < 6{uj). This akeady proves 

Step 2: Suppose (13.2p holds. Fix a, /3 G ^. We deduce from (13. 2p that there exists some N ^JJ" 
such that 

for any s G Q n [t,T], l{^[a]<s} = l{7r[,3]<.} for u e {a =[t^,) /3} \ N. (3.3) 
Fix Lo e n\N. For any s € Q n [t, e{u})], we have u e {a =it,s) /?}• Then ([3:3]) yields 

lWa]<s}(^^) = l{.[^]<.}(u;), for all s e Qn [t,e{uj)]. (3.4) 

If 7r[a]((x)) < 9{uj), take an increasing sequence C Qn [f, 0{uj)] such that s„ t vr[a](a;). Then 

(|3.4|) implies 7r[/3](a;) > s„ for all n, and thus 7r[/3](a;) > 7r[a](a;). Similarly, by taking a decreasing 
sequence {r„}„gN C Qn [t,9{ijj)] such that r„ J, 7r[a](a;), we see from (j3.4p that 7r[/3] < r„ for all n, 
and thus 7r[/3](a;) < 7r[a](a;). We therefore conclude 7r[/?](a;) = 7r[a](a;). Now, if 7r[/3](a;) < 9{uj), we 
may argue as above to show that 7r[a](a;) = 7r[/3](6t;). This proves (j3.ip . which means ir £ Yllj,. □ 

Let us now look at the second scenario in which the controller acts first. In this case, the stopper 
has access to not only the path of W up to time t but also the controller's decision. The controller, 
however, does not use strategies as an attempt to offset the advantage held by the stopper. As 
the next remark explains, the controller merely chooses one single control because she would not 
benefit from using non-anticipating strategies. 

Remark 3.2. Fix t £ [0, T] . Let j : T At satisfy the following non-anticipativity condition: for 
any ti,T2 £ T and s G [t,T], it holds for F-a.e. io £ Q that 

if mm{Ti{u}),T2{u))} > s, then {-f[Ti])r{uj) = (7[r2])r(w) for r £ [t,s). 

Then, observe that 7[r](cj) = 'y[T]{uj) on [t, t{uj)) F-a.s. for any t £ T. This implies that employing 
the strategy 7 has the same effect as employing the control 7[T]. In other words, the controller would 
not benefit from using non- anticipating strategies. 

Now, we are ready to introduce the upper and lower value functions of the game of control and 
stopping. For {t,x) £ [0, T] x M'^, if the stopper acts first, the associated value function is given by 

(3.5) 



U{t, x) := inf sup E 

'rGn*j,c«GA IJt 



On the other hand, if the controller acts first, the associated value function is given by 



V{t, x) := sup inf E 



. (3.6) 



By definition, we have U >V. We therefore call U the upper value function, and V the lower value 
function. We say the game has a value if these two functions coincide. 
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Remark 3.3. In a game with two controllers (see e.g. \13\ [12\ [T^ [9]J, upper and lower value 
functions are also introduced. However, since both of the controllers use strategies, it is difficult to 
tell, just from the definitions, whether one of the value functions is larger than the other (despite 
their names). In contrast, in a controller- stopper game, only the stopper uses strategies, thanks to 
Remark \3.2l We therefore get U > V for free, which turns out to be a crucial relation in the PDE 
characterization for the value of the game. 

We assume that the cost functions /, g and the discount rate c satisfy the following conditions: 
/ : [0, T] X M'' X M I— )• M4. is Borel measurable, and f{t, x, u) is continuous in (x, u), and continuous 



in X uniformly in u £ M for each t; g : 



is continuous; c : [0, T] x 



1-^ 



is continuous 



and bounded above by some real number c > 0. Moreover, we impose the following polynomial 
growth condition on / and g 

\f{t,x,u)\ + \g{x)\ <K{l + \xf) for some p> 1. (3.7) 

Remark 3.4. For any {t, x) G [0, T] x M*^ and a £ A, the polynomial growth condition ()3.7p and 
(|2.5p imply that 



E 



t<r<T \Jt 



< 00. 



(3.8) 



Lemma 3.1. Fix a E A and (s,x) G [0,T] x M'^. For any {(s„,x„)}„gN C [0, T] x M"^ such that 
{sn,Xn) — >• {s,x), wc havc 



E 



sup |(7(X^^"'^"'") - 5(X,"'^'") 
o<r<r 



0; 



(3.9) 



E r|l[.„,T](r-)/(r,X,^-^"'",a,) - l[,,r](r)/(r, X,^'"'", a.)|dr ^ 0. (3.10) 
Jo 



Proof. In view of (j2.7p . we have, for any p > 1, 



E 



sup 



0. 



(3.11) 



Thanks to the above convergence and the polynomial growth condition ()3.7p on /, we observe that 
(|3.10p is a consequence of [Ml Lemma 2.7.6]. 

It remains to prove (|3.9p . Fix e, r/ > 0. Take a > large enough such that '^^''-'^(^+1^1) ^ 2^ 
where Ci > is given as in Remark 12. 4i Since g is continuous, it is uniformly continuous on 
Ba{x) := {y G I |y — x| < a}. Thus, there exists some 6 > such that \g{x) — g{y)\ < e for all 
x,y & Ba{x) with |x — y| < 5. Define 



A := < sup -x\> a>, Bn ■= i sup IX^"'""'"'" - x\ > a 

t 0<r<T J t 0<r<T 

:= I sup - Xn\ > Dn:=l 

[_ 0<r<T ^ J I, 



Xn\>-}, Dn:=\ sup - X','^'^\ > 6 



}■ 



By the Markov inequality and (j2.6 
CiVTil + \x\) 11 



r{A) < 



,, 2Ci\/r(l + |x„|) r? ^ , 
< P(S^) < — ^ < i for n large enough. 
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On the other hand, p. lip implies that P(L'„) < ^ for n large enough. Noting that {B'^Y C for 
n large enough, we obtain 



sup - g{X, 

0<r<T 



>e] < 1 - P(^'= nB^n Dl) = F{A U Bn U 

< P(A U B'„ U D„) < r], for n large enough. 



Thus, we have hn := supo<^<j, [(/(X*"'^"'") — (7(Xr'^'")| — )• in probabihty. Finally, observing 
that the polynomial growth condition (13.70 on g and ()2.5p imply that {/injneN is L^-bounded, we 
conclude that /i„ — )• in L^, which gives (j3.9p . □ 



3.1. The Associated Hamiltonian. For ^) e [0,T] x 

following Hamiltonian with our mixed control/stopping problem: 



where 



H{t,x,p,A) := inf H''it,x,p,A), 



1. 



X M*^ X M"^, we associate the 

(3.12) 



H''{t,x,p,A) := -b{t,x,a) - p- -Tr[aa'{t,x,a)A] - f{t,x,a). 



Note that the operator H is upper semicontinuous as an infimum of a collection of continuous 
functions. It may fail to be continuous, however, as M is only a separable metric space with- 
out any compactness assumption. As a result, we will need to consider the corresponding lower 
semicontinuous envelope i/*, defined as in ()1.2p . in some cases (e.g. Proposition 15. 2p . 



3.2. Reduction to the Mayer Form. Given t G [0, T] and a G At, let us increase the state 
process to (A, Y, Z) , where 

^yM,j/,« = -y^*'^'f'"c(s, A*'^'")ds, s G [t,T], with = y > 0; 

■.= z + J' Y,^''''y'''f{r, A^'^'", ar)dr, for some z > 0. 

Set 5 := M'^ X M+ x M+. For any x := (x, y, z) G 5, we define 

/ A*'"'" 

and consider the function F : S IR+ defined by 

F{x,y,z) := z + yg{x). 
Now, we introduce the functions C/, y : [0, T] x 5 i— )• M defined by 



Uit,x,y,z):= inf sup E F(A^f ,^ rg^'^ Z^f *^'") = inf sup E F(X^;:;p 

y(t,x,2/,z) := sup inf E [F(A*'^'", Y^*'^'?^'", Z*'^'?^'^'")] = sup inf E[F(Xj.''''")]. 

Given r G 7t,T; consider the function 

J(t,x;a,r) := E[F(X*''^'")]. 



(3.13) 
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Observing that F(Xr'") = z + yF(Xr' ' '"), we have 

J{t, x; a, r) = z + yJ{t, {x, 1, 0); a, r), (3-14) 

which in particular implies 

U{t,x,y,z) = z + yU{t,x) V{t,x,y, z) = z + yV{t,x). (3.15) 
Thus, we can express the value functions U and V as 

U{t,x)= inf sup J(t, (x, 1, 0); a, 7r[a]), V{t,x)= sup inf J(t, (x, 1, 0); a, r). 

The following result will be useful throughout this paper. 

Lemma 3.2. Fix (t,x) € [0,T] x 5 and a £ A. For any 9 G Tt,T CLnd r S 7e,T! we have 

E[F(Xt'^'") I = j(0(a;),X*'^'"(a;);a^'",T^'") , /orP-a.e. w G f). 

Proof. See Appendix IA.4I □ 

3.3. e-optimal saddle points. Fix t G [0,T], x = {x,y,z) G S and a G At- Thanks to the 
classical theory of Snell envelopes (see e.g. Appendix D in [19j), the optimal stopping problem 
G"(t, x) := inf^g^^ E[F(Xr^'")] admits an optimal stopping time. More precisely, by the estimate 
([3:81) and [19, Theorem D.12], we have 

G°(t,x) = E[F(Xt'?'°)], where := inf{s > t \ G(s,X*''''") = ^(X*'^'")}. (3.16) 

An immediate consequence is the existence of a saddle point of e-optimal choices. 

Proposition 3.2. Fix e > 0. Given {t, x) G [0, T] x M.'^, there exist a pair {a*, it*) G x 11^ j, such 
that for any a £ At and vr G ^tT' 



E 



F(x: 



n* \a] > 



£ < E 



F(X*f;''°'"* 



< E 



F(X*f'i;°'"* 



Proof. In view of (13.16p . V{t,x) = sup^^g^j E[F(X*'q'^''''")]. Thus, by choosing a* G At such that 
V{t,x) — e < E[F(X^'|^;^''^'" )] and defining vr* G 11* by Tr*[a] = r" Va G A, this proposition 
follows. □ 

Remark 3.5. Presumably, one could obtain a real saddle point (instead of an e-optimal one), by 
imposing additional assumptions. For example, in the one- dimensional game in [20j, a saddle point 
is constructed under additional assumptions on the cost function and the dynamics of the state 
process (see (6.1)- (6. 3) in [2Q]j. For the multi- dimensional case, in order to find a saddle point, 
[23] assumes that the cost function and the drift coefficient are continuous with respect to the control 
variable, and an associated Hamiltonian always attains its infimum (see (71)- (73) in [23] J; whereas 
[IS] and [TU] require compactness of the control set. 

In this paper, we are not interested in imposing additional assumptions for constructing saddle 
points. Instead, we intend to investigate whether the game has a value and how we can characterize 
this value if it exists, under a rather general set-up (while knowing that e-optimal saddle points 
always exist). 
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4. SUPERSOLUTION PROPERTY OF V 

In this section, we will first study the following two functions 

G"(s,x):= inf J(s,x;a,r), G"(s,x):= inf J(s,x;a,r), for (s, x) G [0, T] x 5, (4.1) 



where a £ ^ is being fixed (note that G" was introduced earlier in Subsection I3.3p . A continuity 
result of enables us to adapt the arguments in [8] to current context. We therefore obtain a 
weak dynamic programming principle (WDPP) for the function V (Proposition 14. 1( ) . which in turn 
leads to the supersolution property of V (Proposition 14. 3p . 

Lemma 4.1. Fix a € A. 

(i) G" is continuous on [0, T] x S. 

(ii) Suppose a £ At for some t G [0,T]. Then = G° on [0,t] x S. As a result, G" is 
continuous on [0,t] x S. 

Proof, (i) For any s £ [0,T] and x = {x,y,z) G S, observe from (|3.14|) that G"(s,x) = z + 
yG°'{s, (x, 1, 0)). Thus, it is enough to prove that G°(s, {x, 1, 0)) is continuous on [0, T] x W^. Also 
note that under (j2.4p . we have 

G"(s,x) = inf J(s,x;a,r)= inf J{s,x]a,T). 

Now, for any {s,x) G [0, T] x take an arbitrary sequence {{sn,Xn)}neN C [0, T] X such that 
{sn,Xn) — )• (s,a:). Then the continuity of G°'{s, {x, 1,0)) can be seen from the following estimation 



G°(s„,(x„,l,0)) -G°(s,(x,l,0)) 



inf E[F(X 



Sn,Xn,l,0,a 



)] - inf E[F(X 



< sup E [|F(X, 



< E 



sup |F(X: 



s„,x„,l,0,a\ 
r J 



s,a;,l,0,Q~j j 

F(X: 



s,x,l,0,a^ 



0, 



where the convergence follows from Lemma l3.ll 

(ii) Suppose a G At for some t G [0, T]. For any (s, x) G [0, t] x S and r G 7^,t, by taking 6 = s 
in Lemma 13.21 we have 



J(s,x;a,T) =E[E[F(X^''''") I J-,](a;)] =E[J(s,x;a,T'''^)] > inf J(s,x;a,r), (4.2) 



where in the second equality we replace a**'"^ by a, thanks to Proposition 12.31 We then conclude 



inf J{s,x;a,T)= inf J(s,x;a,r), 



t€Ts,t 



as the "<" relation is trivial. That is, G"(s,x) = G"(s,x). 



(4.3) 



□ 



Now, we want to modify the arguments in the proof of [H Theorem 3.5] to get a weak dynamic 
programming principle for V. Given w : [0, T] x M'^ i— )• M, we mimic the relation between V and V 
in i^J5\i and define u) : [0, T] x 5 M by 



w{t, x, y, z) := z + yw{t, x), (t, x, y, z) G [0, T] x S. 



(4.4) 
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Proposition 4.1. Fix (t,x) G [0, T] x S and e > 0. Take arbitrary a G At, G 7^*^ and 
if G USC([0, T] X M'^) with if <V . We have the following: 

(i) E[(^+(0,Xf'")]<oo; 

(ii) //, moreover, £[(^^(6*, X^^'")] < oo, then there exists a* G At with a* = Us for s G [0,6*) 
such that 

E[F(X*'-'"*)] > ny'r^r^ir A 0, 4xT) + ^^'aY'^'I - 4e, Vr G 7^^. 

Proof, (i) First, observe that for any x = {x, y, z) G 5, (p{t, x) = yip{t, x) + z < yV{t, x) + z < 
yg{x) + z, which implies (p'^{t,-x) < yg{x) + z. It follows that 

the right-hand-side is integrable as a result of (|3.8p . 

(ii) For each (s,r/) G [0, T] x 5, by the definition of y, there exists a^^'^'^'^ G such that 

inf J(s, ??; a("■'')■^ r) > V{s, rj) - e. (4.5) 

Note that 99 G USC([0,r] x M'^) implies cp G USC([0,r] x 5). Then by the upper semicontinuity of 
(f on [0,T] X 5 and the lower semicontinuity of QO'^^''^^'^ on [0,s] x S (from Lemma l4.ll (ii)). there 
must exist r*^*''') > such that 

<^{t',x') - ^{s,ri) < e and G"'''"^'' (s, ??) - G"^'''''''(t', x') < e for all {t',x') G 5(s, r?; r^^-")), 

where S(s,7?;r) := {{t',x') G [0, T] x 5 | t' G (s - r, s], |x' - 7?| < r}, for (s,r/) G [0, T] x 5 and r > 0. 
It follows that if {t',x') G B{s,r];r^^'^'>), we have 

^ ri)-e> V{s, r])-2e> <^{s, t]) - 2e > ^{t' , x') - 3e, 

where the second inequality is due to (|4.5|) . Here, we do not use the usual topology induced by 
balls of the form Br{s,T]) := {{t',x') G [0,T] x 5 | |t' — s| < r,\x' — ri\ < r}; instead, for the 
time variable, we consider the topology induced by half-closed intervals on [0,T], i.e. the so-called 
upper limit topology (see e.g. [HI Ex.4 on p. 66]). Note from [Til Ex.3 on p. 174] and [26', Ex.3 
on p. 192] that (0,T] is a Lindelof space under this topology. It follows that, under this setting, 
{B{s, r/; r) | (s, r/) G [0, T] x 5, < r < r(^'^)} forms an open covering of (0, T] x 5, and there exists 
a countable subcovering {B{ti, x^; rj)}^^^ of (0, T] x 5. Now set ^0 := {T} x S, C-i := and define 
for ah z G N U {0} 

Aj+i := S(ti+i,Xi+i;rj+i) \ Q, where Ci := Ci__i U Ai. 
Under this construction, we have 

{e, X*''''") G U,gNu{0}^i IP-a.s., Ailr^Aj = for i 7^ j, and G"''' (t', x) > (p{t' , x')-2,e for {t\ x') G Ai, 

(4.6) 

where a*'"^ := q(*"^»)''^. 
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For any n G N, set := Uo<i<n^i and define 

a^'" := al[o,,)+ (al(A")c(^,Xf'") + f;a*'n^,(e,Xf'")) l[e,T] e A- 



i=0 



Note that af"' = for s G [0,^). Whenever u> G {(0,Xg'^'") G Ai}, observe that (a 
a^'"- (w (g>e </'e('^')) = '^e <Pei^')) = ia'''^f''^{uj'); also, we have a*'*^ G A(^), as a*'"^ G A, and 

6{ui) < ti on Ai. We then deduce from Lemma [321 Proposition [231 and (|4.6|) that for P-a.e. a; G ri 



i=0 

n 



t,x,a\ 



(4.7) 



i=0 



Hence, we have 



E[F(X:.'-'-^'")] = E[F(X*'-'°)l{,<e}] + E[F(Xf '""")l{,>e}] 

E[F(Xt'^'")l{,<e}] + E [E[F(Xt'^'-'")l{,>,}|J-e]Un(0,Xf 



+ E 



E[F(X*''^'""")l|,>,||J-,]l(^„)e(e,Xf'") 



(4.8) 



> E[F(X*,'-'°)1{,<,}] + E[l{,>,}(^(0,Xf '°)Un(0,X, 



.-t,x,o\ 



3e 



> E[1|,<,|(^(t, X*''''")] + nkr>9}V{0, Xf '")Un (e, Xf '")] - 3e, 
where the first inequahty comes from (14.7P , and the second inequahty is due to the observation that 



Since E[(^+(0, Xg'^'")] < oo (by part (i)), there exists n* G N such that 

E[(^+(e,Xf '")] -E[^+(0,Xf ■")l^„.(0,Xf '")] < e. 

We observe the fohowing holds for any r G T^j^ 

E[l{,>,}^+(0,Xf'")]-E[l{,>,}^+(0,Xf'")l^„.(0,Xf'")] 

< E[^+(0,Xf '")] -E[^+(e,Xf '")l^„.(0,Xf '")] < e. 

Suppose E[<^~ (6*, Xg '''")] < oo, then we can conclude from (j4.9p that for any r G T^j^ 

E[l{,>,}^(0,Xf '")] =E[l{,>,|^+(0,Xf '")] -E[l|,>,|(^-(0,Xf '")] 

< E[l{,>,}^+(0, Xf '°)1^„. (0, Xf '")] + e - E[l{,>,}^-(e, Xf {6, Xf '")] 

= E[l|,>,|(^(e,Xf '")l^„*(0,Xf '")] +e. (4.10) 



(4.9) 



□ 
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Taking a* = a^'" , we now conclude from (|4.8p and (j4.10p that 

We still need the following property of V in order to obtain the supersolution property. 
Proposition 4.2. For any {t,x) G [0,T] x W^, V{t,x) = sup„g^ (x, 1, 0)). 

Proof. Thanks to Lemma 14.11 (ii) , we immediately have 

V{t,x)= sup G"(t,(x,l,0)) = sup G"(t,(x,l,0)) < supG"(t,(x,l,0)). 

a&At a&At aeA 

For the reverse inequality, fix a G ^ and x G 5. By a calculation similar to (14. 2p . we have 
J{t,yL;a,T) = E[J(t, x; a*''^, r*'"^)], for any r E 7^,t- Observing that r*'"^ e 7^*^ for ah r E 7^,t 
(by Proposition [221), and that E[J(t, x; a*''^, r*''^)] = E[J(t, x; q*'"^, r)] for aU r G T^'j. (by Proposi- 
tion [27T]), we obtain 

inf J(t,x;a,r) = inf E[J(t, x; a*'"^, r*'"')] = inf E[J(t, x; a*'"^, r)] 
< sup inf E[J(t, x; a, r)] = sup inf J(t,x;a,r), 

where the inequality is due to the fact that a*'*^ E By setting x := (x, 1,0) and taking supremum 
over a E .4, we get sup„g_4 G"(t, (x, 1, 0)) < y(t, x). □ 

Corollary 4.1. V E LSC([0,r] x M^). 

Proof. By Proposition 14.21 and Lemma 14.11 (i), y is a supremum of a collection of continuous 
functions defined on [0, T] x R*^, and thus has to be lower semicontinuous on the same space. □ 

Now, we are ready to present the main result of this section. Recall that the operator H is 
defined in ([312]) . 

Proposition 4.3. The function V is a lower semicontinuous viscosity supersolution to the obstacle 
problem of a Hamilton-, J acobi- Bellman equation 

ma.xi^c{t,x)w - ^ + H{t,x,D^w,Dlw), w - g{x)^ = on [0,T) x , (4.11) 

and satisfies the polynomial growth condition: there exists N > such that 

\V{t,x)\ < N{1 + |x|^^), V(t,x) E [0,r] X M'^. (4.12) 

Proof. The lower semicontinuity of V was shown in Corollary 14.11 Observe that < V{t, x) < 
sup^g_4^ E[F(X^^'^''''")] < supQ,g_4 E[F(X^^'"^''''")] =: u(t,x). Since v satisfies (|4.12p as a result of 
Theorem 3.1.5], so does V. 
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To prove the supersolution property, let h G C"'^'^([0,T) x M'^) be such that 

= (y-/i)(to,xo) < {V -h){t,x), for any G [0,r) x {t,x) ^ (to,xo), 

for some {to^xo) G [0, T) x W^. If V{tQ,XQ) = g{xQ), then there is nothing to prove. We, therefore, 
assume that V{tQ,xo) < g{xo). For such (tQ^xo) it is enough to prove the following inequality: 

dh 

< cito,xo)h{to,xo) - -^{to,xo) + H{-,D^h,Dlh){to,xo). (4.13) 

Assume the contrary. Then there must exist Co ^ such that 

dh 

> c{to,xo)h{to,xo) - —ito,xo) + H^%-,D^h,Dlh){to,xo). (4.14) 

Moreover, we can choose some r > with to + r < T such that 

> c{t,x)h{t,x) - + H^''{-,D^h,Dlh){t,x), for ah {t,x) G Br{to,xo). (4.15) 

Define C £ -4 by setting Q = Co for all t > 0, and introduce the stopping time 

6 := inf {s > to I {s,Xl°'-^'<) ^ i?.(to,xo)} G 

Note that we have 9 G 7^°^ as the control C is by definition independent of Tto ■ Now, by applying 
the product rule of stochastic calculus to Ys°'^°'^''^h{s, xl'^'^'^''^) and recalling (j4.15p and c < c, we 
obtain that for any r G T^^t^ 

V{to,xo) = h{to,xo) = E •^'^/i(0 A r,X*«;^"°'^) 

+ Yto,-oX( ^ch- — + H^o {.,D.,h, Dlh) + /J (s, Co)^^ 

<E y,*°f'^'^/i(0AT,X*°X^^°'^)+ / yi«'-0'i'^/(s,X*0'-0'^,Co)ds . (4.16) 
L Jto J 

In the following, we will work towards a contradiction to (|4.16p . First, define 

/i(0,X*°'^'°'^'°'^) :=yg*°''^»'^''^/i(0,X*"'"^"'^) + / y/0'^'«'i'^/(s,X*»'^0'^,Co)f^s. 

Note from (14.160 that E[/i(^, Xg"'^"'^'^''')] is bounded from below. It follows from this fact that 
E[/i-(6',X*'''''"'^'°''^)] < oo, as we already have E[/i+(6i, X*°''^°'^'°''^)] < oo from Proposition O (i) • 
For each n G N, we can therefore apply Proposition l4.1l (ii) and conclude that there exists a*'" G At^, 
with as'" = Cs for all s < 6, such that for any r G T^^x^ 



E[F(Xt"''^"'^'°'"*'")] > E 



y;°f "^•^/i(0 A T, 'f) + / yi"'-«'i'^/(s, Co)ds 



to 



Next, from the definition of V and (j3.16p . we have 

V{to,x^) > G-*'"(to,(xo,l,0)) =E[F(Xt«„'"«'i'°''^*'")], 

where 

:= inf {s > to | G"*'" (s, X*«'^0'i'0'°*'") = 5(X*«'^»'°*'")} G 7;J,°2.. 



1 

n 

(4.17) 



(4.18) 
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Combining (j4.18p and (j4.17p . we obtain 



V{to,xo)>K 



to 



1 

n 



By sending n to infinity and using Fatou's Lemma, we conclude that 



V{to,xo)>E 



to 



where r* := liminf„_!.oo t" is a stopping time in T^^rp^ thanks to the right continuity of the filtration 
F*o. The above inequality, however, contradicts (j4.16p . □ 

5. SuBSOLUTiON Property of U* 

As in Sectional we will first prove a continuity result (Lemma l5.4p . which leads to a weak dynamic 
programming principle for U (Proposition 15. ip . Then, we will show that the subsolution property 
of U* follows from this weak dynamic programming principle (Proposition [52]) • Remember that 
U* is the upper semicontinuous envelope of U defined as in ()1.2p . 

Fix s G [0,T] and ^ G LF^{VL,Fs) for some p G [l,oo). For any a & A and '/ri,7r2 G 11^ with 
7ri[/3] < TT2[fi] P-a.s. for all /? G At, we define 



■={l3eA\Pu = au for u G [s,7ri[a]) P-a.s.}, 



and introduce the random variable 
i^^'^'"(^i,vr2) : = 



^ u 

TTi [a] 



esssup E 

Observe from the definition of B^f''^ and ( 13.111 that 



J" 



TTI [a] 



(5.1) 



(5.2) 



(5.3) 



7ri[/3] = TTi[a] P-a.s. V/3 G i3^f 

This in particular implies vr2[/3] > vri[/3] = ^i[a] P-a.s. V/3 G S^f'", which shows that if ^'^'"(vri, vra) 
is well-defined. Given any constant strategies 7ri[-] = n G Tfrp and 7r2[-] = r2 G T^'^j-, we will simply 
write if '■''"(vTi, 7r2) as if ''>'"(ri, r2). For the particular case where ^ = x G M"^, we also consider 

r^'"'°(vri,7r2) := r^"V„^'-'i'"/(n, X^'^ ajdtx + y;;f;;f'°if'^'-'°(7ri, 7r2). 

J s 

Remark 5.1. Let us write K'^'^'°'{-ki,tt2) = esssup^ g^j.^^:, a E[i?^\^^(/3) | Jvi[a]] for simplicity. Note 
that the set of random variables {E[i?^'^^^(/3) | J^ni[a]]} /seBp'"'" closed under pair wise maximiza- 
tion. Indeed, given /Si, P2 G S^f'", set A := {ElR^f^^^Wi) I -^7ri[a]] > ^[Rnf'n2W2) \ ^TTi[a]]} ^ -^7ri[a] 
and define (3^ := Al[o,7ri[a]) + Wi'^A + /?2lA0l[7ri[Q],T] S ;B^f T/ien, observe that 

= EK;^;"^(/3i) I i -^-ih]- 

T/ius, i(;e conclude from Theorem A. 3 in [121 Appendix A] that there exists a sequence {/3"}n6N 
^^f'° such that ir'^'-'°(7ri,7r2) =t lim,^ooE[i?^\^f2(/3") I -^^iH] ^-a.s. 
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Lemma 5.1. Fix (s,x) G [0, T] x M'^ and a £ A. For any r G [s,T] and tt £11^^, we have 



Proof. For any (3 G Bp^'"', we see from Remark 12.51 (i) that Xu^ = 
It follows from (15.21) that 



(r, vr) W-a.s. 



iC*'''''°(r, vr) = esssup E 



Observing from ([51]) that iSr'""'" C ^ = we conclude iv:^'^'<^(r, vr) < K^-^''"'"'"(r, vr). 

On the other hand, for any /3 G define /3 := aljo^^) + /?l[r,r] ^ Bp^'". Then, by Remark 12.51 (i) 
again, we have Xp^ = Xu for u G [r,T] P-a.s. Also, we have 7r[/3] = 7r[/3], thanks to ()3.ip . 

Therefore, 

E 



E 



Y 



In view of ([521), this implies /s:'''^''"'"'"(r, vr) < K'^'^'"(r, vr). 



□ 



Lemma 5.2. Fix (s, x) G [0, T] x M'^. Given a£ A and 'JTi,'JT2, G 11* j, with 7Ti[(3] < tt2[P] < T^sif^] 
P-a.s. for all (3 G As, it holds P-a.s. that 



E 



7ri[a] 



J" 

^ 7 



7ri[a] 



<y;;;;;'"K*'-'"(vri,vr3). 



Moreover, we have the following supermartingale property: 

Er'-'"(^2,vr3) I < r'"'"K,^3) P-«.s. 

Proof. By Remark 15.11 there exists a sequence {/3"}neN in BP2'" such that K*'^'"(vr2, vrs) =t 
(/3") I -7v2[o]] IP-a-S- From the definition of '"^ in ()5.ip . = for u G [s,7r2[a]) 
P-a.s. We can then compute as follows: 



E 



^;'S"^^""(-2,vr3) 



.J 7 



7ri[a] 



E<^ Y 



s,x,l,a 



TT2 [Ol\ 



lim E 

n— >-oo 



7^2 [o] 



^ 7r3[/3"] 



•^772 \a] 



T 



TTi [a] 



E<! lim E 



hm E 

n— >-oo 



I, 



Y 



s,a;,l,, 



J" 



J" 



7ri[«J 



J" 



71"! [Oj 
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where the last hne fohows from the monotone convergence theorem and the tower property for 
conditional expectations. We therefore conclude that 

f"T2[o] 



E 



\ J[u,X^' ,au)du + Y^^^. K [Tr2, its) 

J TTi [a] 



hm E 

n— ^00 



7ri[o] 



7ri[Q] 



',x,l3" on 



J 7 



y^'f'l'" lim E 



TTi [a] 



J" 



where the inequality follows from the fact that /3" G Bnf'°' Q B^f'". It then follows that 



i.x.l,a , 



-E 



P1T2 [a] 
TTI [a] 



TTi[a\ 



< 



K j{u,X^' ,au)du + Y^^^^-^ K (7ri,7r3) = i (vrijTra 



□ 



Lemma 5.3. For any {t, x) G [0, T]xS andTT £ H* j,, 



sup J(i, x; a, 7r[a]) = sup J{t,x; a,7r[a]). 



Proof. Fix a £ A and x G 5. For any vr G 11^ j,, by taking = t in Lemma 13.21 we have 

Jit,x;a,TT[a]) =e\e[F{X';^;^) \ Tt]{io)\ = E x; a*'", 7r[a])] < sup J(t, x; a, 7r[a]), 

where in the second equality we replace 7r[a]*''^ by vr[a], thanks to Proposition 12.11 The lemma is 
then a consequence of the above inequality. □ 

Now, we are ready to state a continuity result for an optimal control problem. 

Lemma 5.4. Fix t G [0, T]. For any vr G the function L'" : [0,t] x S defined by L'^(s,x) : = 

supQ,g_4_^ J(s, x; a, 7r[a]) is continuous. 

Proof. Observing from (j3.14p that L'^{s, x) = yL'^{s, {x, 1, 0))+z, it is enough to show the continuity 
of L''(s, (x, 1, 0)) in (s, x) on [0, t] x W^. By [SI Theorem 3.2.2], we know that J{s, {x, 1, 0); a, r) is 
continuous in x uniformly with respect to s G [0, t], a G .4, and r G 71, t- This shows that the map 
{s,x,a) I— )■ J{s, {x, 1, 0); a, 7r[a]) is continuous in x uniformly with respect to s G [0,t] and a £ A. 
Then, we see from the following estimation 

sup |L'^(s, (x, 1, 0)) — L'^(s, (x', 1, 0))| < sup sup |J(s, (x, 1, 0); a, 7r[a]) — J(s, (x', 1, 0); a, 7r[a])| 

that L^{s, (x, 1,0)) is continuous in x uniformly with respect to s G [0,t]. Thus, it suffices to prove 
that L^{s, (x, 1,0)) is continuous in s for each fixed x. To this end, we will first derive a dynamic 
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programming principle for L'^{s, (x, 1, 0)), which corresponds to [24^ Theorem 3.3.6]; the rest of the 
proof will then follow from the same argument in [24], Lemma 3.3.7]. 

Fix {s,x) E [0,t] X R'^. Observe from (fSl^) that Bp""'" = A for ah a G A. In view of (fO]) . 
this implies that i('*'^'"(s, vr) = esssup^g^ E[F(X^'|^'j^'°'^) | J^s], which is independent of a G ^. 
We will therefore drop the superscript a in the rest of the proof. Now, we claim that K'^'^(s,it) is 
deterministic and equal to L'^{s, {x, 1,0)). First, since r G T^j, implies that r G Tfrp for all s G [0, t], 
we observe from Lemma 13.21 Proposition 12.11 (ii). and Proposition 12.31 that 



K^'^'is,!:) > esssupE[F(X^',^',^'°'°) | Ts]{-) = esssup J(s, (x, 1, 0); a"'', 7r[a]''-) 
= sup J{s, {x, 1,0); a, TT[a]) = L'^{s,{x,l,0)). 

aeAs 



(5.4) 



On the other hand, in view of Remark 15.11 there exists a sequence {a^}n£n in A such that 
ir"'^(s,7r) =tlim„^ooK[F(X^'j^'^j°'"") | J^,] P-a.s. By the monotone convergence theorem, 

E[K^'^{s,7r)] = E [ lim E[F(X^'5^j°'"") | J",]! = lim E[F(X^'PY'"")] 



<supE[F(X^'pi'°'")] = L-(.,(x,l,0)), 



(5.5) 



where the last equality is due to Lemma lS^ From (j5.4p and (j5.5p . we get K^'^{s, vr) = L^{s, (x, 1, 0)). 
Then, for any a £ A, thanks to the supermartingale property introduced in Lemma 15.21 we have 
for all r G [s,t] that 

L-(s,(x,l,0)) = K''-{s,n) = r''^'"{s,n) > E[r^'-'"(r,7r)] > E[r^'-'-(7r, ^)] > E[F(X^'p,i'°'°)], 

where the last equahty follows from the fact that i^^'^''^(7r, tt) = esssup^^^s,x,a ^(X^j^j^) > g{X^'^^) 
ir-a.s., as a result of (|5.2p . By taking supremum over a £ A and using Lemma 15.31 we obtain the 
following dynamic programming principle for ^"^(5, (x, 1,0)): for all r G [s,t], 

L^(s,(x,l,0)) = supE[r"'^'"(r,7r)] 

aeA 



sup E 

a€A 



1^ y„^'^'^'"/(n, au)du + y/'-'i'°L-(r, {X^^'^, 1,0)) 



where the second equality follows from the fact if*'^'"(r, vr) = K"^'-^^' '"(r, vr) = L'^(r, (X*'^'", 1,0)) 
P-a.s., as a consequence of Lemma l5.11 Now, we may apply the same argument in [24, Lemma 3.3.7] 
to show that L'^{s, (x, 1,0)) is continuous in s on [0,t]. □ 

Proposition 5.1. Fix (t,x) G [0,r] x S and e > 0. For any vr G Ul j. and ip G LSC([0,r] x M^) 
with f > U, there exists vr* G 11^ such that 



E 



F(x*f;"' 

7r* [a 



< E 



( 7r[a],X'f'^ ] + Z^f f •^•'^ + 4e, Va G A- 



t,x,y,z,a 

nla] 



Proof. For each (s, r/) G [0, T] x 5, by the definition of U, there exists vr^'*''^)'^ ^H^j, such that 



sup J {s,rj; a, vr^'"''')'^ [a] ) < U{s, rj) + e. 
a^Aa 



(5.6) 
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Recall the definition of in ([33]) and note that tp G LSC([0,r] x W^) implies (p G LSC([0,r] x S). 
Then, by the lower semicontinuity of (p on [0, T] x 5 and the upper semicontinuity of L'^ ' on 
[0, s] X 5 (from Lemma l5.4p . there must exist r > such that 

p{t',x') - p{s,T]) > -e and L"''''"''(t', x') - L"''''"''(s, r/) < e, 

for any {t',x') contained in the ball B(s,r];A^'^^) defined as in Proposition 14.11 It follows that if 
it',x') G B{s,r];A^''^^), we have 

L"*''"^'' {t', x') < L"*''"'' {s, r])+e<U{s,r]) + 2e < p{s, r,) + 2e< ip{t', x') + 3e, 

where the second inequality is due to (j5.6p . By the same construction in the proof of Proposition 14. H 
there exists a countable covering x^; rj)}jgN of (0, T] x S, from which we can take a countable 

disjoint covering {^i}iGNu{o} of (0,T] x S such that 

(7r[a],X^|'^P G UieNu{0}^i ^-a.s. Va G A and L""(t',x') < p{t',x') + 3e for {t',x') G ^i, (5.7) 

where vr*'^ := vr^*"^*)'^. 

For each n G N, set A" := Uo<i<n ^« and define vr^'" G ^ by 

n 

7r^'"[a] := ri(^.).(vrH,X:;|'„p + J]vr^'^[a]U(7r[a],X^Jp, Va G ^. 

1=0 

Now, fix a £ At- For G |(7r[a], X^|^p G Ajj, observe that 

(vr-'"[a])-M'-(a;') = vr^'^H (c ®^[,] </<^[,](u;')) = ^*''H = (vr^'^[a])-[-l''^(a;'), 

and 7r*'^[a] G T^1q"|[^Vj as 7r*'^[a] G 7^*V and 7r[a](tj) < tj. We then deduce from Lemma [37 
Proposition I2.1|, and (|5.7|) that for P-a.e. G 0, 



E|F(X*;,'^f 



7r[Q] 



(w) lA"(7r[a](w),X*;p\"(w)) 



< 



< 



5]j(^H(^),X*;|'„pa;);a-["l'-,7r^'^[a]) U(vr[a](a;), X^f'^pa;)) 

1=0 
n 



i=0 



\^ip (n[a]{u;),X.lj-^^{u;)^ + 3e U-(vr[a](w), X*;|'^°(a;)). 



It follows that 

.-t,x,a 



E 



= E 

< E 

< E 



E 

+ E 



U-(vr[a],X;[^ 



E 



7r[Q:] 



l(^n)c(7r[a],X^|^P 



V3(7r [a] , X*; J;" ) (vr [a] , X^J 



(/:3(7r[a],X*; 



t,x,c^^ 



+ 3e + E 



+ 3e + E 

t,x,a 



F(X^'''")l(An)c(7r[a],X^J 



t,x,a\ 



^,x,a^ 



(5.8) 



where the last inequality follows from the fact that > > 0. Since E[F(X,j^ ' )] < oo by 
()3.8p . from the monotone convergence theorem we can take some n(a) G N large enough such that 
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E 



F(X^'''")l(^„(„)).(7r[a],X^J'^P < e. Finally, by defining n* G Uy as 7r*[a] := 7r"'"(")[a] Va G 



we conclude from (15. 8p that 



E 



•-t,x,a\ 



+ 4e = E 



1 03 vr a , A r , ] + Z \ 



7t,x,y,z,a 



+ 4e. 



□ 



The following is the main result of this section. Recall that the operator H is defined in p.l2p . 
and denotes the lower semicontinuous envelope of H defined as in (jl.2p . 

Proposition 5.2. The function U* is a viscosity subsolution to the obstacle problem of a Hamilton- 
Jacobi- Bellman equation 

max I c{t, x)w - ^ + H*{t, x, Dr^w, DIw),w - g{x) \ = on [0, T) x M*^, 

and satisfies the polynomial growth condition (|4.12p . 

Proof. We may argue as in the proof of Proposition 14.31 to show that U* satisfies ()4.12p . To 
prove the subsolution property, we assume the contrary that there exist h S C^'^([0, T) x M*^) and 
{to,xo) e [0,T) X R'^ satisfying 

= ([/* - h){to,xo) > {U* - h){t,x), for any {t,x) G [0,T) x {t,x) / {to,xo), 

such that 

meixi^c{to,xo)h{to,xo) - ^{to,xo) + H^{-,Dr,h,Dlh){to,xo),h{to,xo) - g{xo)^ > 0. (5.9) 

Since U*{to,xo) = h{to,xo) and U < g hy definition, continuity of g implies that h{to,xo) = 
U*{to,xo) < g{xo). Therefore, we can conclude from ()5.9p that 

dh 

c{to,xo)h{to,XQ) - —{to,xo) + H^{-, Dxh, Dlh){to, xo) > 0. 
Define the function h by 



h{t,x) := h{t,x) +£{\t - top + \x- xq\^). 

Note that {h, dfh, D^h, D'^h){to, xq) = {h, dth, D^h, D'^h){tQ, xq). Then, by the lower semicontinuity 
of H^: , there exist some e > and r > with to + r < T such that 

dh 



c{t,x)h{t,x) - ^{t,x) + H"-{-,D.^h,Dlh){t,x) > 0, V a e M and {t,x) e Br{to,xo). (5.10) 



Now define rj > hy 



Tje 



cT 



min [h — h) > 0. 



(5.11) 



dBr{t(),XQ) 

Take (t,x) G Br{tQ,xo) such that \{U — /i)(t,x)| < rj/2. For each a G Aj., define the stopping time 



r := inf {s > i I (s,4'^'") i Br{to,xo)] G 7^^. 
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Note that we have 0" G 7^*^ because the control a is independent of T^. Applying the product rule 



of stochastic calculus to y^* '^'^'"/i(s, Xs'^'"), we get 



h(i, x) = E 



> E 



dh 



c/i - ^ + + / (5, a,)ds 



where the inequality follows from (jS.lip . (|5.1U|) and c < c. Moreover, by our choice of {t,x), we 
have U{t,x) + ri/2 > h{t,x). It follows that 



U{t, x) > E 



+ |, for any a G Af. (5.12) 



Now, define vr G Ilt^ by 7r[a] := 0" for all a £ Af:, and recall the strategy vr* G H*^ introduced in 



Proposition 15.11 Then, from the definition of U and Proposition 15. H there exists a G ^£ such that 



U{i,x) = U{i,x, 1,0) < sup E 

a€Af 



F 



< E 



FX 



-I 



< E 



^ 2' 



which contradicts (I5.12p . 



□ 



6. Comparison 

In this section, to state an appropriate comparison result, we assume a stronger version of 
as follows: there exists K > such that for any t, s G [0, T], x,y £ M.'^, and u G M, 

\b{t, x, u) — b{s, y,u)\ + \cr{t, x, u) — a{s, y,u)\ < K{\t — s\ + \x — y\). 

Moreover, we impose an additional condition on /: 

f{t,x,u) is uniformly continuous in {t,x), uniformly in n G M. 



(6.1) 



■2) 



Note that the conditions (j6.ip and ()6.2p . together with the linear growth condition ()2.3p on b and 
a, imply that the operator H defined in (|3.12p is continuous, and thus H = H:^. 

Proposition 6.1. Assume (j6.ip and ()6.2p . Let u (resp. v) be an upper semicontinuous viscosity 
subsolution (resp. a lower semicontinuous viscosity supers olution) with polynomial growth condition 
to dilT]) . such that u{T, x) < v{T, x) for all x G M"'. Then u < v on [0, T) x M*^. 

Proof. For A > 0, define := e^^u{t,x), := e^^v{t,x), and 

Hxit, x,p, A) := inf | -b{t, x,a)-p - l-Tr[aa'{t, x, a)A] - e^^f{t, x,a) \ . 

aGM [2 J 



Note that the conditions (j6.ip and (j6.2p . together with the linear growth condition (j2.3p on b and a 
and the polynomial growth condition ()3.7p on /, imply that /f;s^ is continuous. By definition, u (resp. 
v) is upper semicontinuous (resp. lower semicontinuous) and has polynomial growth. Moreover, by 
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direct calculations, the subsolution property of u (resp. supersolution property of v) implies that 
(resp. v^) is a viscosity subsolution (resp. viscosiy supersolution) to 

Tiiayiiy{c{t,x) + X)w-^ + Hx{t,x,D:^w,Dlw), w-e^^g{x)^ = on [0,r) x M'^. (6.3) 

For any {t, x, r, q,p, ^) G [0, T] x M"* x M x E x M'^ x M"*, define 

Fi{t, x, r, q,p, A) := (c(t, x) + X)r — q + Hx{t, x,p, A) and -^2(^5 x, r) := r — e^^g{x). 

Since Fi and F2 are by definition continuous, so is -F3 := maxji^i, i^2}- We can then write (j6.3p as 
F^{t,x,w,^,D^w,Dlw)=Q. 

Prom the polynomial growth condition on and v^, there exists some p > Q such that 

\u^{t,x)\ + \v^{t,x)\ 
sup < 00. 

Define 7(x) := 1 + and set ip{t,x) := e~^^'y{x). From the linear growth condition (|2.3p on 
b and a, a direct calculation shows that \b{t,x,a) ■ D^j + ^Tr[aa'{t,x,a)D'^j]\ < C'j{x) for some 
C > 0. It follows that 

dip r 1 1 

(c(t, x) + X)(f — h inf < —b{t, X, a)Dxip — -Tr[aa' {t^ x, a)D'^ip] > 

ot a£M [2 J 

[c{t, x) + 2X]-y + inf \ -b{t,x,a)D:^-f - l;Tr[aa'{t,x,a)Dlj] 

^ agA/ 2 



(6.4) 



> e~^*[c(t, x) + 2A - C]7 > 0, if A > ^. 



Now, take A > ^ and define := v"^ + £ip for all e > 0. By definition, is lower semi- 
continuous. Given any h G C^'2([0,r) x R'^) and (to, 2:0) G [0,T) x M*^ such that - h at- 
tains a local minimum, which equals 0, at {tQ,xo), the supersolution property of implies either 
Fi {■,h{-),^{-),D^h{-),Dlh{-)) {to,xo) > or F2(-,/i(-)) (to, 2^0) > 0. If the former holds true, we 
see from (|6.4|) that 

Fi (•, v^-), ^(0, a.v^{-),Dlv^{-)^ (to, xo) > 0; 

if the latter holds true, then F2 (•,v^(-)) (^0,2:0) = v^{to,xo) - e^^°g{xo) = F2 {■,v^{-)) {to,xo) + 
Sip{to,xo) = F2 (•,/i(-)) (to,xo) + eip{to,xo) > 0. Therefore, is a lower semicontinuous viscosity 
supersolution to ()6.3p . 

We would like to show u"^ < on [0,T) x M'^ for all e > 0; then by sending e to 0, we can 
conclude n < w on [0, T) x M.'^, as desired. We will argue by contradiction, and thus assume that 

:= sup {u^ - fe)(t,x) > 

[0,T]xR<^ 

From the polynomial growth condition on if^ and v'^ and the definition of ip, we have 

lim sup(u'*' — v^){t,x) = —00. 
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It follows that there exists some bounded open set O C such that the maximum N is attained 
at some point contained in [0, T] x O. For each 6 > 0, define the functions 

^s{t, s, X, y) ■= u^{t, x) - v^{s, y) - ris{t, s, x, y), with r]s{t, s, x, y) := ^[\t - + \x - y\^]. 

Since is upper semicontinuous, it attains its maximum, denoted by A''^, on the compact set 
[0, T]^ X at some point {ts, ss,xs,y5)- Then, the upper semicontinuity of u^{t,x) — v^{s,y) 
implies that (^u^{ts,xs) — v^{ss,y5)) ^ is bounded above; moreover, it is also bounded below as 

N < Ns = u^{ts, xs) - v^{ss, ys) - mits, ss, xs, ys) < u^{ts, xs) - v^{ss, ys)- (6.5) 

Then we see from (j6.5p and the boundedness of [u''^{ts,xs) — v^{ss,y5)) ^ that {ris{ts,ss,xs,ys))s 
is also bounded. Now, note that the bounded sequence {ts,ss,xs,ys)s converges, up to a subse- 
quence, to some point {i,s,x,y) G [0, T]^ x . Then the definition of 775 and the boundedness of 
{ri5{ts^ ss-:Xs.,ys))s imply that t = s and x = y. Then, by sending 5 to in (j6.5p . we see that the 
last expression becomes {u^ — v^){i,x) < N, which implies that 

Ns^ N and r]s{ts, ss, xs, ys) 0. (6.6) 

In view of Ishii's Lemma (see e.g. |29l Lemma 4.4.6]) and |29^ Remark 4.4.9], for each 6 > 0, 
there exist As,Bs G M*^ such that 

Tr(CC'As - DD'Bs) < -AC - D\^ for ah C,De W^, (6.7) 



and 

\{ts - ss),^ixs -ys),As^ e P'^'^u^{ts,xs), (^^{ts - ss),^{xs - ys), Bs^ e P^'~v^{ss,ys), 

where P'^'~^w{t,x) (resp. P'^'~'w{t, x)) denotes the superjet (resp. subjet) of an upper semicontin- 
uous (resp. a lower semicontinuous) function w at (t, x) G [0, T] x M'^; for the definition of these 
notions, see e.g. [10] and [29]. Since the function F3 = max{Fi,F2} is continuous, we may apply 
[29; Lemma 4.4.5] and obtain that 

max |(c(t5,X5) + X)u^{ts,xs) - ^{ts - ss) + Hx{ts,xs, ^{xs - ys),As), u^{ts,xs) - e^^^g{xs)^ < 0, 

max|(c(s5,y5) + X)v^{ss,ys) - ^{ts - ss) + Hx{ss,ys, ^{xs - ys),Bs), v^{ss,ys) - e^^'^giys)^ > 0. 
Noting that max{a, 6} — max{c, d} > min{a — c,b — d} for any a, b,c,d £ M, we then have 

mini {c{ts,xs) + X)u^{ts,xs) - {c{ss,ys) + X)v^{ss,ys) + Hx{ts,xs, ^{xs - ys),As) 

1 1 ^'-'^ 

- Hxiss, ys, -^{xs - ys),Bs), u^{ts, xs) - v^{ss,ys) + e^''g{ys) - e^*'g{xs) \ < 0. 

Since u^{ts, xs) - v^ss, ys) + e^^^g{ys) - e^^^g{xs) = Ns + rjs{ts, ss,xs,ys) + e^^^giys) - e^^^g{xs) ^ 
> 0, we conclude from (j6.8p that as 6 small enough, we must have 

{c{ts,xs) + A) u^{ts, xs) - {c{ss,ys) + A) v^{ss, ys) 

11 3 

< Hxiss, ys, -^{xs -ys),Bs) - Hx{ts,xs, -^{xs - ys),As) < fi{\ts - ss\ + \xs - ys\ + -^\xs - ysl"^), 
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for some function such that fj,{z) — )• as z — t- 0; note that the second inequaUty follows from 
(lO) . dO]) . and dSZl). Finally, by sending 5 to and using dSj]), we get {c{i,x) + A)iV < 0, a 
contradiction. □ 



Now, we turn to the behavior of 14, the lower semicontinuous envelope of V defined as in (II. 2p . 
at terminal time T. 

Lemma 6.1. For all x € W^, V^{T,x) > g{x). 

Proof. Fix a ^ A. Take an arbitrary sequence {tm,Xm) — ^ iT,x) with tm < T for all m G N. By 
the definition of y, we can choose for each m G N a stopping time Tm G T^^^y such that 



V{tm,Xm) > inf E 



> E 



/•Tm 



1 

m 



Note that Tm, ^ T as Tm ^ TT*™^ and — ^ Then it follows from Fatou's lemma that 
liminfm->oo ^(i ) > g{x). Since (t 

rri) ^^m.) is arbitrarily chosen, we conclude V^(T', > 9ix)- D 

Theorem 6.1. Assume ^Jij and (fOj) . T/ien ^7* = y on [0,r] x M'^. /n particular, U = V on 
[0, T] X M°', i.e. t/ie game has a value, which is the unique viscosity solution to (|4.1ip with terminal 
condition w{T,x) = g{x) for x G M"^. 

Proof Since by definition U{t,x) < g{x) on [0,r] x M^, we have U*{t,x) < g{x) on [0, T] x 
by the continuity of g. Then by Lemma 16.11 and the fact that U* > U > V > V.^, we have 
U*{T,x) = V{T,x) = g{x) for all x G W^. Recah that under and the function F 

is continuous, and thus H = H^. Now, in view of Propositions 14.31 and 15.21 and the fact that 
U*{T,-) = V{T,-) and H = H*, we conclude from Proposition O that U* = V on [0,T] x M"', 
which in particular implies U = V on [0, T] x M!^. □ 

Appendix A. Proofs for Sections [2] and [3] 

This Appendix is devoted to rigorous proofs of Propositions 12.11 12.21 12.31 and Lemma 13.21 To 
this end, we will first derive several auxiliary results. 

Recall the definitions introduced in Subsection 12.11 Fix t G [0,T]. For any ^ C Jl, ^ C f]*, and 
x G M*^, we set 

Ar, := {uj G A \ u}t = x}, 

and define 

A*'"^ := {u \uj^t(^ A}, A^^"^ := {A*''^)^, u A := {u (g)t u \ u £ A}. 
Given a random time r : $7 i— t- [0,oo], whenever cj G ^2 is fixed, we simplify our notation as 

^r,a; ^ j]^r{u),u ^ g^^g^ COUSidcr 

nl■.= ^P^'Gl'''^gl, VsG[t,T]. (A.l) 
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Note that the inclusion follows from the Borel measurability of ipf Finally, while E denotes the 
expectation taken under P, in this appendix we also consider Ep, the expectation taken under P. 

Lemma A.l. Fix t £ [0, T] and uj £ fl. For any r £ [t, T], AeQr, A£ gj., and ^ G L°(0, Gr), 

(i) A^;"^ = ^o'"" + X and A^^'^ G g*'^', Vx G M^. 

(ii) A*'"^ = V'r^^o'^ e "^r ^ a^t^ P*(^*'^) = P'-^X^^'"^) = P'-^X^*'"^). Vx G M'^. 

(iii) ^^^A*'"^ G (Ar^'^r ^ and F{(j)^^A^'^) = P*(yl*''^). 

(iv) w 0t A^^t £ ^r- • Hence, co 0t £ Qt ■ 

(v) For any Borel subset £ ofR, {^^'"^y^iS) G (A^^'^r ^ ^r- i^ence, ^^'"^ G ^^(l^,^^). 

Proo/. (i) Fix x G M'^. Since oj G ylo"^ «4> w (g>t cj G ^ and = «4> (w (cu + x)). = a;.l[o,t](-) + 
((w. + x) — + x) + a;t)l(j^T] (•) = {uj (g)^ tD). G A and (w + x)t = x lD + x G Ax'^ , we conclude 

Set A:= {Acn \ J^i" G ^*'^}. Note that G A since 17^''^ = {tt; G 17* | w G SI, tD* = x} = 
(5^*)x e ^r^. Given ^ G A, we have (A^)!;"^ = (17*)^\{6D G 17* | w 0* G ^, cDt = x} = (17*)^ V^^'"^ G 
^r^, which shows A'^ G A. Given {yljlig^ C A, we have (UieN^*)!'^ ~ UjgNi'^ G 17* | (g)^ w G 
Ai,iJjt = x} = \Ji^fq{Ai)x^ G ^r^, which shows UjeN ^* ^ ^- Thus, we conclude A is a ci-algebra 
of Q. For any x G Q'^ and A G Q+, the set of positive rationals, let Ox{x) denote the open ball 
in centered at x with radius A. Note from \18\ p. 307] that for each s G [0, T], Q^. is countably 
generated by 



Cr ■■= I rHWfXHOxSx^)) m G N, G Q, s < ti < ••• < t„ < r, x, G A, G Q+|. 
i=i ^ 



(A.2) 



Given C = r\T=iiWuyHOx,{xi)) in Cr = C^, if > t, set k = min{i = 1, ■ ■ ■ ,m \ U > t}; 



r j 



otherwise, set A; = m + 1. Then, if ojt^ ^ 0\^{xi) for some i = 1, • • • ,k — 1, we have C^:' = G ^. 
if A; = m + 1 and wj. G OA^l^^i) Vi = 1, • • • , m, we have Cl''^ = (17*)x G ^; for all other cases, 

m 

c*'- = {w,' = x}n [}{wl)-' {O^M -^t + x)) G g'f. (A.3) 



i=k 



Thus, Cr ^ A, which implies Qr = c(Cr) ^ A. Now, for any A G Gr, Ax^ G C ^*. 

(ii) Observe from part (i) that w G A*'"^ <^ G (J - (Dt G ^g'"^ i.e. ipt{oj) G Aq''^ <^ a; G 
ipyiA^^'^). Thus, ^*''^ = Vt"^(^o'^) e = ^ thanks to part (i) and (HH). Then, 
using part (i) again, P*(y4*''^) = P*(A*''^) = P*'^(A*''^ + x) = P*'^(y4^''^) = f^^^^A^^^), Vx G M'*. 

(iii) By part (ii) and the Borel measurability of (j)t : (17,^,.) i— )• (17*,^*), we immediately have 
(p^A^''^ G (l)t^ni C g^. Now, by property (e") in pj, p. 84] and part (ii). 



U*'^ I gt+](tj') = p*.^t(A*'^) = P*(y4*''^) for P-a.e. u' G 17, 
which implies P[</>7^A*''^] = P*(^*''^). 
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(iv) Set A := {A Q \ u} (S>t A^^ G Gr}. Let C* be given as in (|A.2p . For any C = 
CliLii^tJ^^^ {Ox^{xi)) in C*, we deduce from the continuity of paths in Q that 

w = {tj' G I = a;^ Vs G Q n [0, t) and tJj^ G Ox^^i) for i = 1, • • • , m} 

Thus, we have C* C A. Given {AJjeN C A, we have uj (g)f (UieN^i)^^* = UieN(^ '^t e ^r, 

which shows |JjGN^« ^ ^' ^'^^^ ™ particular impHes 0* = UneN(^r) ""^(^"(0)) ^ ^- Given A G A, 
we have w (8)^ (^^)cjt = {(^ '^t \ '<^t ^uit) ^ which shows A*^ G A. Hence, A is a ci-algebra 

of $7*, which imphes = cr(C*) C A. Now, by part (i), we must have uj (Sh At^^ G Qr- 

(v) Since e fi^r, = W en \ C{uj (g)t (Pt{uj')) e £} = W e n \ uj(g)t M^') e 

^-\£)} = ^^\^-^£)Y^^ G (pt^ni C g,,, thanks to part (iii). □ 

In hght of Theorem 1.3.4 and equation (1.3.15) in |31| . for any G-stopping time r, there ex- 
ists a family {Q'^}ujen of probability measures on {^1,Qt), called a regular conditional probability 
distribution (r.c.p.d.) of P given Qr, such that 

(i) for each A £ Gt, the mapping uj i— )• Q^iA) is ^T-measurable. 

(ii) for each A G Qt, it holds for P-a.e. uj e n that ¥[A \ ar](w) = Qr{A). 

(iii) for each uj e n, Q'^ {uj (^r (f^^^""^)^,) = 1- 

By property (iii) above and Lemma I A. II (iv), for any fixed w G ri, we can define a probability 
measure Q^'^ on , ) by 

Q-.-(i) ■.= Q'^{uj (^r i^J, Vi G 

Then, combining properties (ii) and (iii) above, we have: for j4 G it holds for P-a.e. uj €z Q that 

F[A I QrKuj) = ((a; ®, (n^^^^U) n ^) = Q!;^(u; 0, vd^v") = Q^''^(^^''^). (A.4) 

Note that the r.c.p.d. {Qr It^^en is generally not unique. For each (t, x) G [0, T] x M'^, observe 
that the shifted Wiener measure P*'^ can be characterized as the unique solution to the martingale 
problem for the operator L := ^ X^f j=i dJ^dx starting from time t with initial value x (see |30t 
Remark 7.1.23] and |3H Exercise 6.7.3]). Then, thanks to the strong Markov property of solutions 
to the martingale problem (see e.g. [31t Theorem 6.2.2]), there exists a particular r.c.p.d. {Qr l^en 
such that Q^'"^ = wH'^^(^), Now, by (|X4l) and Lemma El] (ii), we have: for A G Gt, 

F[A I gr]{uj) = p^H."rM(^T,c^) = prH(^r,a;^^ p_^_g^ 

So far, we have restricted ourselves to G-stopping times. We say a random variable r : i— )• [0, oo] 
is a G-optional time if {r < t} G Gt for all t G [0,r]. In the following, we obtain a generalized 
version of (jA.5p for G-optional times. 

Lemma A. 2. Fix a G-optional time t <T. For any A G Gt, 

F[A I Gr+]{^) = P^('^)(^^''^) for P-a.e. w G 0. 
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Proof. Step 1: By [181 Problem 1.2.24], we can take a sequence {r,i}„gN of G-stopping times such 
that Tn{ui) I t{ui) for all u e ^. Fix A e Gt- For each n G N, (lA.Sp implies that for any B G 

Ep[U1b] =Ep[P^"('^)(^^"''^)1b]. (A.6) 

Then, for any B G ^t+; we must have ()A.6P for all n G N, since Gr+ = HnGN^T-n- ^ow, by taking 
the limit in n and assuming that for each u & Q 

lim f>M^)(^A^n,i^) = P^('^)(A^''^), (A. 7) 

n— )-oo 

we obtain from the dominated convergence theorem that Ep[1a1s] = Ep[P^('^)(^^''^)1b]. bmce 
B G is arbitrary, we conclude | ^r+](w) = P^(^)(^^'^) for P-a.e. uj £ Q. 

Step 2: It remains to prove (IA.7p . Fix oj £ fl and set A := {A C Q | (jA.7p holds}. Since 
J]^.'^ = J]'*, Vs G [0, T], (jXTll holds for and thus J] G A. Given ^ G A, we have p^nH [(^c^)r„,a;] ^ 

pT„Hj^^T„,w-jcj _ ]^ _ pT„(a;)^^T„,w-) _^ _ pT(i^)(^^T,tj-) _ pT{w)^^^T,a;^cj _ pT{w) ^^^c^T,wj ^ which 

shows A"^ G A. Given a sequence of disjoint sets in A, observe that {A^''^}i^fq is a sequence 

of disjoint sets in for any s G [0,r]. Then we have P^"^'^H(U»gn ^*)^"''^] = 1"^"^'^-' [U»gn A^"'"^] = 
E.eNP^"^"HA"'") ^ E.gnIP^^^HA"'") = IP^^^HU.gnA'I = IP^^")[(U.6nA)^'1- which shows 
UiGN A ^ ^- Thus, we conclude that A is a u-algebra of Q. 

As mentioned in the proof of Lemma lA. 1 1 (i) . Qt is countably generated by Ct = Cj. given in ()A.2p . 
Given C = nili(^t»)"HCA, (a^i)) ™ ^T, if > t{uj) we set k := mm{i = 1, - ■ ■ ,m \ U > r(w)}; 
otherwise, set k := m + 1. We see that: 1. If cjf. ^ 0\^{xi) for some i = I, • • • , /c — I, then C"^''^ = 
Vs G [r(tj),T] and thus ()A.7p holds for C. 2. If /c = m + 1 and 0;^. G 0(xj) for all i = 1, • • • , m, 
we have C^'^ = Jl'* Vs G [T(a;),T] and thus (|X77ll still holds for C. 3. For aU other cases, Ct'f is of 
the form in ()A.3P Vs G [T(a;),r]. Let 5 be a d-dimensional Brownian motion defined on any given 
filtered probability space {E,I, {Xs}s>0) P)- Then by Lemma [A. II (ii), 

= G Oa. (x. - a;,„(^)), i = • • • ,m] 

^ P[Bu-riu.) G Oa.(x. - = A;- • • ,m] = p-M,-.(.) [C^;-] = P-H[C-.-]. 

Hence, we conclude that Ct ^ A and therefore Qt = cf{Ct) ^ A. □ 

Now, we want to generalize Lemma lA.ll to incorporate F-stopping times. 

Lemma A. 3. Fix 6 £ T- We have 

(i) For any 7V G AT, jf'"^ G AT^^"^^ and ^iV^'"' G Jf for P-a.e. oj e n. 

(ii) For any r G [0,T] and ^ G J>, /loZds /or P-a.e. a; G i7 t/iat 

< r, A^'- G -Hf(-) U AT'^") C and (^^'a''^ G -F.^^^). 

(iii) For any r G [0,T] and ^ G J>), it holds for P-a.e. w G that 

if6{oj)<r, e^'"^ G L°(l^,J-,^(")). 

Proo/. (i) Take iV G TV such that N C N. By [ig Exercise 2.7.11], there exists a G-optional 
time r such that A^i := / r} G A/". By Lemma IA.21 there exists N2 £ M C J7 such that 
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= F[N I Gr+Kuj) = P^H(Ar^.^), for w G \ N2. Thus, for a; G \ (iVi U N2), we have 

On the other hand, from Lemma |Al] (m) , f{(t>g^N^''^) = P^('^)(iVS'^) = for cj G J] \ (Fi UiVa), 
which shows 0^ ^A^^'"^ G P-a.s. Since c/)"^]?^''^ C cpg^N^'"^, we conclude (/-"^iV^''^ G TV" P-a.s. 

(ii) By [18, Problem 2.7.3], there exist i G ^r- and iV G AT such that A = AuNandAnN = 9. 
From Lemma lA.ll (ii). we know that for any G if e{uj) < r then A^^^ G V.r^'^^ C Q^^'^K Also, 
from part (i) we have A^ ' G AA P-a.s. We therefore conclude that for P-a.e. uj £ if 9{oj) < r, 
then A^'^ = i^'"^ UiV^''^ G Tit^'^^ uJf^^'^'^ C ^^^'^^ Then, thanks to part (i) and Definition EH it 
holds P-a.s. that (pQ^A^''^ = <i)-^A<^^^ U (p-Q^jf'"" G qy^'^rLr^'^^ UM ^ Tr^'^^ if 6(10) < r. 

(iii) Let <S be a Borel subset of M. Since ^^^{£) G J>, we see from part (ii) that, for P-a.e. w G SI, 
(^^'-)"' {£) = Wen\ i{uj®eh{^')) G £:} = K G I uj^ehiu^') e i-\£)] = ^.^rH-?))''" e 
J^^"^^ if e{io) <r. □ 

Now, we generalize Lemma lA. 21 to incorporate F-stopping times. 

Lemma A.4. Fix 9eT. For any A G Tt, | -T^eK^) = F^^'^\a^''^), for P-a.e. cj G 0. 

Proof. Thanks again to [181 Exercise 2.7.11], we may take a G-optional time r such that A^i := 
{9 ^ t} £ J\f and J> = Tg. Moreover, we have A = U A^ for some A £ Qt and N £ J\f with 
^ n A^ = 0, by using [18^ Exercise 2.7.3]. Then, in view of Lemma lA.ll (ii). Lemma lA.31 (i), and 
Lemma I A. 2 1 we can take some A^2 G AT such that for u £n \ {Ni U A^2), 

^s{ui),^e,uj^^ = P^^'^V^'^''^) = P'^'-'^Va^''^) -FP^^'^ViV^''^) = P^H(A^'^) 

_ (A.8) 
= P[i I gr+]{co) = F[A I gr+]{u) = ¥[A I Gr+Koo). 

For any S G J>, = S U iV' for some B £ Qr ^ Qt+ and W' £j\f with 5 fl iv' = 0, thanks again 
to da Exercise 2.7.3]. We then deduce from ^EB that E[l^lij] = E[l^l^] = E [p^^'^V^'")l^ = 

E [p^^''V''^)1b1 • Hence, we conclude P^^'^V^'"') = ^[^ I -^t](w) = ^A \ J^e]{uj), foi uj £ n\ 
{N1UN2). □ 

Finally, we are able to generalize Lemma lA.ll (iii) to incorporate F-stopping times. 

Proposition A.l. Fix 6 £ T. We have 

(i) for any A £ Tt, P[v1 | Fe\{u:) = P[(/)g ^^^'"], for P-a.e. w G 

(ii) for any ^ £ Li(l^, J"t, P), E[^ | J"e](^^) = E [^^'''^j /or P-a.e. oj £ Q. 

Proof, (i) By Lemma lA. 31 (i) and Lemma lA.ll fiii). it holds P-a.s. that 

The desired result then follows from the above equality and Lemma IA.4I 

(ii) Given A £ Tt^ observe that for any fixed a; G fi, (lyi)^''^(Lt;') = \A{y> ®e <i)d^^')) = 
l^-i^0_^(u'). Then we see immediately from part (i) that part (ii) is true for ^ = 1a- It fol- 
lows that part (ii) also holds true for any J"x'-measurable simple function ^. For any positive 
^ G L^{^,Tt,^), we can take a sequence {^njnGN of -Fr-measurable simple functions such that 
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t '^(w) Vw G rj. By the monotone convergence theorem, there exists N £ J7 such that 
E[^„ I Te]iuj) t I -FeKcj), for a; G Q \ A^. For each n G N, since ^„ is an J-^^-measurable simple 
function, there exists Nn G AT such that E[^ri | J^e]{^) = E [(Cn)^'"^] , for a; G ft\Nn- Finally, noting 
that there exists n' £j\f such that ^^■'^ is J-r-measurable for ci; G \ A^' (from Lemma [A.SI (iii)) 
and that ('^n)^''^(w') t \/uj' G (from the everywhere convergence ^„ t 0? we obtain from 

the monotone convergence theorem again that for (j G 17 \ ( (UneN ^n) U N U n' 



E[C I TeKoj) = lim E[^n \ m^^) = 1™ E[{Cnf^^] = E[^'^^]. 
The same result holds true for any general ^ G L^{Q,,J-t,^) as = — □ 
A.l. Proof of Proposition [27T1 

Proof, (i) Set A := {A Q n \ ¥{A D B) = F{A)¥{B) MB G Ft]. It can be checked that A is a 
a-algebra of VL. Take A G ^^T^?^^ ^ AT. If ^ G AT, it is trivial that ^ G A; if ^ = ^^"^C with 
C G 7^^, then for any B £ Ft., 

¥{A nB) = F{B n (pt^C) = E [P(5 n ^^^C | Ft)] = E [P(5 n (/^^-^C | J"t)(t^)lij(a;)] . 

By Proposition E3](i), for P-a.e. uj e FiBHcp^^C \ Ft){u;) = F[(j)-\B ncp^^CY^''] = P[0i"^C] = 
P(^) if a; G -B. We therefore have F{A n B) = F{A)F{B), and conclude A e A. It follows that 
(pJ^'Hip U7V C A, which implies F^ = a{(l)^^'Htp U7V) C A. Thus, F^ and Ft are independent. 

(ii) Let A denote the set operation of symmetric difference. Set A := {A C ^} | ((j)^^ A^'^)AA G 
J\f for P-a.e. oj G 0,}. It can be checked that A is a cr- algebra of n. Take A G (/)i"^?^^U7V. If A G AA, 
we see from Lemma [01 (i) that A G A; if yl = (j)^^C with C G then = cp^^C = A for 

ah uj eO,, and thus A G A. We then conclude that F^ = a{(i)^^%tp U AT) C A. 

Take a sequence {^n} of random variables in L^iVi^Fj^) taking countably many values {rjjjgi^ 
such that ^n{^) — ^ ^{^) for all w G il. This everywhere convergence implies that for any fixed w G f^, 
(Cn)*''^(w') ^ i^^^'iuj') for aU w' G Vl. Now, fix n G N. For each i G N, since (Cn)"^^^} e C A, 
there exists N"^ G A/" such that for cj G O \ iV^ , 

([(Cn)*'1"'{r.}) A(^„)-HrO = [4>i^ {{in)-\r,}f^] ^{inY^in} =: mT G AT, (A.9) 

where the first equality follows from the calculation in the proof of Lemma [A.3l (iii). Then, we deduce 
from (M that: for any fixed w G 17 \ UieN^' {^uY'^^W) = in{^') for aU w' G f7 \ U»eN^- I* 
follows that: for any fixed a; G 17 \ Ui,„eN^r, (^n)*''^(^^') = in{^') for all G 17 \ Ui,„GN^r and 
n G N. Setting iV = Ut,„eN^r and M = Ui,neN^' obtain that for any G 17 \ iV, 

^(w') = lim in{^') = lim (C„)*'"'(6^') = e*''^(w'), for 6<j' G 17 \ M. 

□ 
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A. 2. Proof of Proposition [2721 

Proof. Take a sequence of stopping times {rjjjgN C Tsuch that takes values in {m/2* | m G N} for 
each i E N and Ti{uj) | t{uj) for all u £ Q (thanks to [ISl Problem 1.2.24]). Set N ■= {t < 9} e Jf. 
Since rj(cj) ^ for all uj €z Q, we have Tj > on \ for all z S N. For each i G N, let 

:= m/2*, m € N. Since {tj < rj„} E J^^,, for all m G N, we deduce from Lemma lA. 31 (ii) and the 
countability of {r'm}meN that there exists gN" such that for a; E ^2 \ A^*, 

if e{oj) < rl, cbe'in < rl^Y'^ € -F^f"^ for all m G N. (A.IO) 

Fix r G [0, T]. For any uj G ri\(]Vu]V*), \ie{uj) > r, then Tj(a;) > e{uj) > r and thus (pg^in < r^^'^ = 
(fig^^ = G Tr^^^; if ^(w) < r, there are two cases: 1. 3 m* G N s.t. r^, G [6'(cj),r] and > r. 

"H^i < C*}^'^ G -F!/"'^ C Tr^""^; 2. 3 m* G N s.t. rj^, < 9{lo) 



Then, by dAlOD, (/.^H^i < ^}''" - ,■^^■m' 

r 

and r^.+i > r. Since Tj(a;) > 6'(tj) > rj„., (/"^^{rj < r}"'"^ 



G-F; 



Thus, for a; G J] \ (iVU we have 4>g^{Ti < r}^'^ G J"r^'^\ and therefore 

{rf'" < r} = {Ti (a; (g)e M^')) < r} = < r}''^ G T^.'^^l V r G [0,r]. 

This shows that rf'"^ G 7^(J;]V ^ S7 \ (77 U 77'). Hence, for w G J7 \ (77 U (U 



N 



have Tj^''^ G T^l^^^j^ Vi G N. Finally, since the filtration F^^*^-* is right-continuous, t^''^{co') 
limj_^oo '''i'^i^') (this is true since Tj | r everywhere) must also be a stopping time in T^^^-^j,. 



we 

=i 
□ 



A. 3. Proof of Proposition 12.31 Recall the metric p on ^ defined in (12. 3p . We say /3 G ^ is a 

step control if there exists a subdivision = to < < ■ " " < = T", m G N, of the interval [0, T] 
such that fit = f^u fo'" * ^ [^i, i«+i) for i = 0, 1, • • • , m — 1. 

Proof. By |241 Lemma 3.2.6], there exist a sequence {a"} of step controls such that a" — )• a. For 
each n G N, in view of Proposition 12.11 (ii), there exist Nn,Mn G M such that: for any fixed 
CO £ n\Nn, (a;!)*'^(w') = <(w') for {r,uj') G [0,T] x (f] \ M„). It follows that: for any fixed 
w G ^\UneN^n> «)*''^(6<j') = <(tj') for ah {r,uj') G [0,T] x (f^\U„eN^n) and n G N. With the 
aid of Proposition lA.il (ii), we obtain 



= lim ^(a",a) = lim E 



hm E I E 

n— >oo 



Tt 



lim 



lim 
lim 

n— >oo 




p'(a;?, ar)dr I (w') (iP(a;') (iP(w) 



p' «)*'"(^^'),ar (^0 f^lP(^ 



p((a")*'"',a*'")(iP(a;) 
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where the last equality is due to the dominated convergence theorem. This implies that 



lim„_^ooP(a",a' 



for P-a.e. u £ Vt. Recalling that a" — >• a, we conclude that p{a 



t,LL) 



a) 



-a.e. 6J G ri. The second assertion follows immediately from [24, Exercise 3.2.4]. 
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A. 4. Proof of Lemma 13.21 

Proof. By taking ^ = F(Xr^'°) in Proposition lA.ll (ii) and using Remark 12.51 (ii), 
E[F(Xt'^'") I J^g]{uj)=K 

= j F (^X'l"i'''«""^"^'"''"(a;')) dn^') = J (^(L^),X*'"'"(a;);Q^'",r''''^) , for P-a.e. Cc; G O. 

□ 
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